Controlling and/or operating a medical device by means of a light pointer

ABSTRACT

The invention relates to a system for controlling and/or operating a medical device ( 30 ) associated with a display ( 20; 50 ) on which medical images ( 24 ) and/or control and/or operating elements ( 21, 22, 25 ) are displayed, wherein the system comprises: —a light pointer ( 10; 40 ) which projects a delimited light cursor ( 14; 44 ); —a light detection device which is associated with the display and comprises a sensor ( 23 ) for determining the presence and location of the light cursor ( 14; 44 ) projected by the light pointer ( 10; 40 ); and—a converter ( 26 ) which converts the captured presence and location information into control and/or operating commands for the medical device. The invention also relates to a method for controlling and/or operating a medical device ( 30 ) associated with a display ( 20; 50 ) on which medical images ( 24 ) and/or control and/or operating elements ( 21, 22, 25 ) are displayed, wherein a delimited light cursor ( 14; 44 ) is projected onto the display ( 20; 50 ) by means of a light pointer ( 10; 40 ), wherein the presence and location on the display ( 20; 50 ) of the light cursor ( 14; 44 ) projected by the light pointer ( 10; 40 ) is detected by means of a sensor ( 23 ), and wherein the captured presence and location information is converted into control and/or operating commands for the medical device ( 30 ).

The present invention relates to controlling and/or operating a medicaldevice by means of a light pointer. In particular, a system forcontrolling and/or operating a medical device and a method for the samepurpose are provided. Operating and/or controlling medical devices, forexample in an operating theater, is often cumbersome or problematic forphysicians in terms of maintaining high levels of sterility. Medicaldevices used in operating theaters such as surgical or medicalnavigation systems, for which the present invention is particularlysuitable, have in the past been operated and/or controlled in variousways, which each have their own problems.

Known control and/or operation systems employ several different modes ofinteraction between a user and a navigation system, including forexample touch screens, keyboards and mouses, voice control or remotecontrol. If touch screens are used (in most cases, the screens used asthe navigation system display), it is necessary to maintain theirsterility by draping them with sterile drapes or by using a steriletouching device such as a sterile pen. Sterile drapes placed over ascreen impair the image quality, in particular when the screen istouched with contaminated hands. Moreover, touching a draped screen withgloved hands can lead to imprecise inputs owing to poor sensualfeedback. Using a sterile pen in conjunction with a display with nodrapes allows for good image quality, but still incurs a number ofgeneral problems. One of these problems is that a user still needs toapproach the display in order to interact with the navigation system andmay thus be forced to leave their working position. Moreover, such pensare expensive and there is still some risk to sterility, for example ifa sterile pen touches an unsterile display or monitor. To avoid such arisk, pens often have to be exchanged a number of times during anoperation.

The use of keyboards and mouses is rather cumbersome and not intuitive,and there are always problems in providing special keyboards and mouseswhich can be kept sterile.

The use of voice control is not generally accepted, on the one handbecause of the necessity to wear a microphone and on the other handbecause of technical problems in identifying spoken commands. Remotecontrols can be used in sterile bags, but do not generally allow forintuitive control.

As briefly discussed above, the present invention is particularly usefuland applicable in connection with surgical or medical navigationsystems, one of which is for example known in general terms from DE 19639 615 A1, which are employed in assisting medical personnel in planningand carrying out medical procedures. Such navigation systems can forexample be used in conjunction with instrument tracking systems in orderto show, on a display, a positional relationship between instruments anda patient's body or parts of a patient's body, images of which have beenacquired beforehand, for example as CT or MR image data sets. Thenavigation system can also be used to guide a user through a sequence ofsteps to be carried out during a medical procedure.

An input apparatus for a computer system using a pointer device isproposed, for different purposes, in U.S. Pat. No. 7,547,025 B2.

It is the object of the present invention to provide a system and methodfor controlling and/or operating a medical device which overcome theaforementioned problems of the controlling and/or operating systems andmethods according to the prior art. In particular, an easy-to-manage wayof controlling and/or operating medical devices is to be proposed whichin particular avoids interruptions to the workflow and/or sterilityproblems.

A medical device as used in conjunction with the present invention canbe referred to in general terms as an operation-assisting medical deviceor a medical device used in an operating theater. Although the presentinvention is mainly explained herein with respect to a medicalnavigation system, it is not limited to such an application or such amedical device. Other medical devices, in particular those used in anoperating theater, can also be controlled and/or operated by employingthe present invention, such as for example any device which can beturned on or off or switched to another mode of operation, such aslamps, movable couches, fluoroscopic devices, etc.

The present invention is defined by claim 1 which relates to a controland/or operating system for a medical device and by claim 12 whichrelates to a method for controlling and/or operating a medical device.The sub-claims define advantageous embodiments of the present invention.

The system according to the present invention controls and/or operates amedical device associated with a display on which medical images and/orcontrol elements and/or operating elements are displayed. The systemcomprises: a light pointer which projects a delimited light cursor; alight detection device which is associated with the display andcomprises a sensor for determining the presence and location of thelight cursor projected by the light pointer; and a converter whichconverts the captured presence and location information into controland/or operating commands for the medical device.

In other words, the system of the invention comprises a light pointer,the pointing spot of which can be detected by a sensitive display, suchthat the medical device is controlled and/or operated, i.e. such thatfor example a virtual button is “pushed” or a certain area on thedisplay is marked and/or highlighted. Thus, the present inventioncontrols a medical device by means of remote control, using a light spotwhich offers a variety of advantages, especially in the field ofapplication for which the present invention is primarily intended. Oneof the main advantages is that an easy-to-manage remote control systemis provided, wherein the controlling and/or operating action can becarried out at the working location of the user—for example, a surgeon.The user does not have to leave the place they are currently working at,in order to approach the display, but can instead “transmit” a “lightcommand”. This enables a fluent working sequence. The sterility of theuser is not affected by controlling and/or operating the device becauseno potentially unsterile input device has to be touched.

In accordance with one embodiment, the light pointer comprises a switch,a multi-deck switch or a number of switches for controlling its lightprojection. Additionally or alternatively, the laser pointer cancomprise switchable or selectable light sources of one or morewavelengths, or a wavelength modifier, as its light source. Additionallyor alternatively, the laser pointer can also comprise a switchable orcontrollable cursor-forming unit for forming the projected light cursor,in particular a light slit or a lens or an object lens. All of the abovefeatures can help to produce and/or modify the light signal emitted by alight pointer, in order to create a variety of different control oroperating commands or inputs. These individual commands can of coursealso complement each other in order to achieve an optimum mode ofcontrol for the medical device.

In a preferred embodiment of the present invention, the light pointer isa laser pointer which produces a finely delimited or formable cursor.Other light pointers can of course also be used, providing they are ableto deliver a well-defined and sufficiently bright light spot.

The light detection device can comprise a photosensitive surface orlayer on the display, in particular on a monitor screen, in particular atouch screen. In technical terms, location detection as employed usingtouch screens or multi-touch screens can be used to set up suchdisplays, while the presence of a light cursor can be detected byphotosensitive elements which are integrated into said system anddetermine the location of the cursor on the display. The screen can ofcourse additionally exhibit touch screen properties, such that acombination of light cursor inputs and touch inputs can be made andused.

The light detection device can be set up in such a way that it comprisesa position-detecting light grid or laser grid covering the surface ofthe display which is, in a preferred embodiment, a monitor screen.Alternatively or additionally, the light detection device can comprise acamera or a camera system which monitors the display.

In accordance with one embodiment of the system of the invention, thedisplay is a projection screen and the system also comprises an imageprojector which projects the medical images and/or the control elementsand/or operating elements of the medical device onto the projectionscreen. This is particularly useful if rather large displays are needed,for example if an ongoing treatment and its image assistance is to beshown to a larger number of people, for example during a demonstrationor presentation.

The display surface can be a light-reflecting surface or can be coveredwith a light-reflecting layer which preferably reflects light at thewavelength emitted by the light pointer. It may also be advantageous tochoose a reflecting surface which solely or mainly reflects saidwavelength, in order to avoid image interference. The surface of amonitor can of course be subjected to a corresponding treatment in orderto acquire said reflection properties.

In accordance with one aspect of the invention, the medical device is amedical or surgical navigation or treatment planning system. When usingsuch a surgical or medical navigation system or treatment planningsystem, there is a need for interaction between the user and thenavigation system, for example in order to effect control inputs. Thiscan include various kinds of control inputs, for example control inputsfor activating a certain procedure or for advancing one step in asequence of navigationally assisted steps in a procedure. Other controland/or operating functions can also be carried out in accordance withthe present invention, such as for example marking and/or highlighting acertain portion of a medical image for registration purposes.

The invention also relates to a method for controlling a medical deviceassociated with a display on which medical images and/or control and/oroperating elements are displayed, wherein a delimited light cursor isprojected onto the display by means of a light pointer, wherein thepresence and location on the display of the light cursor projected bythe light pointer is detected by means of a sensor, and wherein thecaptured presence and location information is converted into controland/or operating commands for the medical device.

The method in accordance with the invention can of course also be usedto control and/or operate a medical device in accordance with theexamples given above, i.e. by using the systems described or parts ofthese systems.

Various other aspects of the method in accordance with the presentinvention involve controlling and/or operating the medical device in oneor more of the following ways:

-   -   a control command or input is effected by pointing the light        cursor at a spot for a predetermined and/or extended period of        time;    -   a control command or input is effected by pointing the light        cursor at a spot, projecting the cursor at a first wavelength in        order to highlight an element on the display and then changing        the wavelength in order to effect a control and/or operating        command with respect to the element;    -   a control command or input is effected by pointing the light        cursor at a spot and then projecting a predetermined succession        of light beams in order to effect a control and/or operating        command with respect to the element;    -   a control command or input is effected by pointing the light        cursor at a spot in order to highlight an element on the display        and then projecting a predetermined succession of light beams in        order to effect a control and/or operating command with respect        to the element;    -   a control command or input is effected by pointing the light        cursor at a spot on the display and then projecting a        predetermined succession of light beams in order to effect a        control and/or operating command with respect to the element;    -   a control command or input is effected by pointing the light        cursor at a spot on the display in order to highlight an element        on the display and then projecting a predetermined succession of        light beams in order to effect a control and/or operating        command with respect to the element;    -   a control command or input is effected by pointing the light        cursor at a spot on the display and then altering the form of        the cursor in order to effect a control and/or operating command        with respect to the element;    -   a control command or input is effected by pointing the light        cursor at a spot on the display in order to highlight an element        on the display and then altering the form of the cursor in order        to effect a control and/or operating command with respect to the        element;    -   the cursor is made visible by highlighting an element on the        display by means of a control software for the display.

Other embodiments of the present invention may involve using one or morecameras of the navigation or tracking system to track the light point orlaser cursor. The light pointer can be integrated into any instrument orcan also be a separate light pointer. It can be attached to aninstrument or any other item used in the medical environment by a clipor other fastening means. It can also be integrated into gloves(surgical gloves), into glasses which are sometimes used by surgeons orinto a head band. The light pointer can be embodied as a single-use,sterile/disposable light pointer. Another embodiment of the lightpointer is provided in a sterile bag which the projected light beam canpenetrate.

In its integrated version, the light pointer can be integrated with avariety of items, for example hardware pointers, other hand-heldinstruments or hand-held parts of navigation systems. One highlyintegrated version would be to use a laser light projector which in itsoriginal function is used as a registration aid for surface-matchingregistration purposes. While it has been discussed above that thedisplay can be reflective, another way of making the location of thecursor visible is also contemplated by the present invention, whereinthe light detection device determines whether the cursor is hitting thedisplay and the graphical software itself highlights this point on thedisplay, such that reflection is not necessary.

The invention will now be explained in greater detail by referring tospecific embodiments. It should be noted that each of the features ofthe present invention as referred to herein can be implementedseparately or in any expedient combination. In the attached drawings:

FIG. 1 depicts a first embodiment of a system in accordance with theinvention, used to highlight a part of a medical image;

FIG. 2 depicts a second embodiment of the system in accordance with theinvention, used to operate and/or control a navigation system;

FIG. 3 depicts a third embodiment of the system in accordance with theinvention, using a light pointer integrated in a hardware pointer; and

FIG. 4 depicts another embodiment of the present invention, using aprojection system.

A first embodiment of a system in accordance with the present inventionis shown in FIG. 1. A light pointer, embodied in this case by a laserpointer 10, is used to project a light beam 13 onto the display 20 of amedical navigation system 30. The laser pointer 10 has two buttons 11and 12 which can be used to influence the properties of the projectedlight beam 13, for example to change its color or wavelength in order totransfer different information/commands for different control and/oroperation purposes. The laser pointer 10 in FIG. 1 can also to seen tocomprise, on its front end, a beam-shaping unit 16 which can consist ofa variable light gap or a lens or an object lens which can alter theform and/or intensity of the beam 13, again for the purpose of effectingvarious different control and/or operating functions. The unit 16 forshaping the beam or changing its intensity or color can be used inconjunction with the above-mentioned functions of the buttons 11 and 12,or the buttons 11 and 12 and the unit 16 can be used separately.

The display 20 can be a monitor on which a display is shown andcontrolled by the navigation system 30 which is connected to the displayby a wired or wireless connection 33 for this purpose. Particular inputelements or “software buttons” 21, 22 are shown on the display 20, aswell as a patient image taken beforehand, which bears the referencenumeral 24. Such a patient image 24 may be one of an image data setproduced by a CT or MR unit.

Reference numeral 23 is intended to show how the surface of the display20 has a certain property which is achieved by surface treatment, byselecting an appropriate surface or by applying an appropriate surfacecover or covering layer. In the present case, the surface 23 is embodiedto be reflective, so that the cursor 14 produced by the light beam 13can be seen on the display 20. The user producing the cursor 14 on thedisplay 20 will then see exactly the location which the cursor 14 ispointing at. In the present example, the cursor 14 is pointing at aparticular landmark on the patient image. Said landmark is for examplehighlighted if the cursor lingers on it for more than three seconds, anda click on a button or a change effected at the unit 16 can then markthe cursor for a special task, for example for registering said landmarkfor navigation purposes. The display 20 is equipped with a lightdetection device which determines the presence and location of the lightcursor 14, and the respective information is collected in a converter 26which converts it into signals which are transferred to the navigationsystem 30 via the connection 32, i.e. the commands and/or inputs issuedby the laser pointer 10 via its cursor 14 on the display 20 are sent tothe navigation system 30 via the converter 26. The navigation system 30can then send back a signal or control command to the display 20 inorder to change the display, for example in order to prompt the surgeonto choose the next registration landmark.

Another embodiment of the present invention is shown in FIG. 2. Itshould be noted that elements in the figures which bear the same orsimilar reference numerals are either the same elements or functionallysimilar elements, for which reason the explanation of their functionswill not be repeated here. There are three main differences between thesystems of FIGS. 1 and 2. One of these differences relates to the designof the button or buttons on the laser pointer 10. The button 15 shown inFIG. 2 is a single button which is however schematically shown to havetwo areas. This is intended to indicate that the button is a two-deckbutton which exhibits a first triggering point and a second triggeringpoint following the first triggering point, comparable to the release ona photographic camera. In the system of FIG. 2, it can be seen that asoftware button 25 shown on the display 20 is being pointed at with thebeam 13 of the laser pointer 10 via the cursor 14. To this end, the userpresses the button 15 down as far as the first triggering point, as aresult of which the laser beam is produced and positions the cursor 14on the software button 25. Once certain that the button 25 has beencorrectly targeted, the user can then press the button 15 down further,as far as the second triggering point, which changes the properties ofthe light beam 13 (for example its color, form, etc.), wherein thischange sends an “activate” command to the display 20. Upon receivingthis command, the software in the navigation system 30 will know thatthe button 25 has been activated and will accordingly carry out aparticular control feature, such as for example proceeding to the nextimage.

Another feature shown in FIG. 2 is the supporting and charging means 31,which in the present example is attached to the navigation system 30.Said supporting and charging means 31 can be used to hold the pointer 10while it is not in use, and to recharge its batteries.

FIG. 3 shows another two features which can be embodied by a system inaccordance with the present invention. FIG. 3 shows that the laserpointer need not be provided as a single, separate unit but may also beprovided as a laser pointer unit 46 which is permanently or removablyattached to a surgical instrument, said instrument (in this case) beinga hardware pointer 40 comprising a pointer tip 45 and a navigation ortracking reference 42. The pointer 40 is again equipped with a button 41for activating a light beam 43 in order to produce a cursor 44. In thepresent example, the cursor 44 is positioned on a target button(software button) 22 on the display 20. The user presses down on thebutton 41 in order to produce a steady beam 43 onto said software button22, for example for three seconds. This is detected by the softwareemployed in the display/navigation system 30, and the button 22 is thenhighlighted by the software in order to indicate to the user that saidbutton is now targeted and ready to be “pressed”. A “pressing” signalcould be produced by the user, also using a single button, for exampleby briefly pressing the button twice, in the manner of a “double-click”.The command issued can then again operate and/or control the entiresystem in a predetermined way—for example, a certain navigationprocedure can be initiated.

A somewhat different embodiment, which however also shows a system inaccordance with the present invention, is depicted in FIG. 4. FIG. 4again shows a laser pointer 10 with a button 11 for triggering a lightbeam 13. However, said light beam 13 is emitted in order to place acursor on a projection screen 50, i.e. on the reflective surface 51 ofsaid projection screen 50. Contrary to the foregoing examples, theprojection screen 50 is a passive element which merely reflectsradiation beamed upon it. It is shown in a side view, hence it is notpossible to see what is being projected onto it but it is possible tosee the arrangement of the other important elements of the system ofFIG. 4, which include a projector 70 which is connected to a medicalnavigation system 30 and which can project images onto the display suchas those shown on the display 20 in FIGS. 1 to 3. To this end, theprojection field 71 of the projector 70 is configured to exactly coverthe projection screen 50 in order to show as large a picture aspossible. This large picture may be particularly expedient if theprocedure being followed is to be communicated to an audience, forexample during a demonstration or presentation.

In order to detect the image reflected by the screen 50 (or itsreflective surface 51), a camera 60 exhibiting a field of view 61 isprovided. The camera 60 is used as a light detection device and “sees”the navigation display on the screen 50, including the cursor 44. Thecamera can then on the one hand communicate this raw image to thenavigation system 30 which can include a converter which converts thepresence and location of the cursor 44 into navigationally usableinformation. On the other hand, such a converter can also be included inthe camera 60, such that pre-processed information can be provided tothe navigation system 30. Detecting the position is made easier if thecamera 60 and projector 70 are arranged such that they have apredetermined positional relationship (i.e. in a calibrated system), forexample a fixed positional relationship such as in FIG. 4, in which theyare integrated into a single unit, although they can be composed of twoseparate parts attached to each other.

The medical navigation system 30, having received the information aboutthe presence of the cursor 14 and its location relative to the projectedimage, can then process commands and control and/or operating inputs inthe same manner as described above with respect to FIGS. 1 to 3 and cancarry out the same or other actions which in turn influence the imageprojected by the projector 70, as indicated in FIG. 4 by the arrow 33which represents a communication line between the navigation system 30and the projector 70.

The projector 70 and the camera 30 can be arranged in accordance withthe available space, but should be fixed once installed, as indicated bythe fixation means 80.

A system as shown in the examples of FIGS. 1 to 4 enables the surgeon towork very time-effectively and to interact with the navigation systemwithout any need to be close to the display. Laser pointers are verycheap and small and can therefore be integrated into almost anyinstrument. The surgeon can exactly locate the laser target point, hencecontrol can be exercised with a high degree of reliability. Any otherlaser-equipped instruments in the operating room can also be used as alaser pointer or light pointer of the present invention, such that thepresent invention can be very easily integrated into an existingoperating room set-up.

1. A system for controlling and/or operating a medical device associatedwith a display on which medical images and/or control and/or operatingelements are displayed, characterized in that the system comprises: alight pointer which projects a delimited light cursor; a light detectiondevice which is associated with the display and comprises a sensor fordetermining the presence and location of the light cursor projected bythe light pointer, and a converter which converts the captured presenceand location information into control and/or operating commands for themedical device.
 2. The system according to claim 1, characterized inthat the light pointer comprises a switch, a multi-deck switch or anumber of switches for controlling its light projection.
 3. The systemaccording to claim 1, characterized in that the light pointer comprisesswitchable or selectable light sources of one or more wavelengths, or awavelength modifier, as its light source.
 4. The system according toclaim 1, characterized in that the light pointer comprises a switchableor controllable cursor-forming unit for forming the projected lightcursor, in particular a light slit or a lens or an object lens.
 5. Thesystem according to claim 1, characterized in that the light pointer isa laser pointer.
 6. The system according to claim 1, characterized inthat the light detection device comprises a photosensitive surface orlayer on the display, in particular on a monitor screen, in particular atouch screen.
 7. The system according to claim 1, characterized in thatthe light detection device comprises a position-detecting light grid orlaser grid covering the surface of the display, in particular a monitorscreen.
 8. The system according to claim 1, characterized in that thelight detection device comprises a camera or camera system whichmonitors the displayer).
 9. The system according to claim 1,characterized in that the display is a projection screen and the systemalso comprises an image projector which projects the medical imagesand/or the control elements and/or operating elements of the medicaldevice onto the projection screen.
 10. The system according to claim 1,characterized in that the display surface is a light-reflecting surfaceor is covered with a light-reflecting layer which preferably reflects,or solely reflects, light at the wavelength emitted by the lightpointer.
 11. The system according to claim 1, characterized in that themedical device is a medical or surgical navigation or treatment planningsystem.
 12. A method for controlling and/or operating a medical deviceassociated with a display on which medical images and/or control and/oroperating elements are displayed, wherein a delimited light cursor isprojected onto the display by means of a light pointer, wherein thepresence and location on the display of the light cursor projected bythe light pointer is detected by means of a sensor, and wherein thecaptured presence and location information is converted into controland/or operating commands for the medical devices).
 13. The methodaccording to claim 12, wherein the system according to claim 2 is usedto control and/or operate the medical device.
 14. The method accordingto claim 13, wherein the medical device is controlled and/or operated inone or more of the following ways: a control command or input iseffected by pointing the light cursor at a spot for a predeterminedand/or extended period of time; a control command or input is effectedby pointing the light cursor at a spot, projecting the cursor at a firstwavelength in order to highlight an element on the display, and thenchanging the wavelength in order to effect a control and/or operatingcommand with respect to the element; a control command or input iseffected by pointing the light cursor at a spot and then projecting apredetermined succession of light beams in order to effect a controland/or operating command with respect to the element; a control commandor input is effected by pointing the light cursor at a spot in order tohighlight an element on the display and then projecting a predeterminedsuccession of light beams in order to effect a control and/or operatingcommand with respect to the element; a control command or input iseffected by pointing the light cursor at a spot on the display and thenprojecting a predetermined succession of light beams in order to effecta control and/or operating command with respect to the element; acontrol command or input is effected by pointing the light cursor at aspot on the display in order to highlight an element on the display andthen projecting a predetermined succession of light beams in order toeffect a control and/or operating command with respect to the element; acontrol command or input is effected by pointing the light cursor at aspot on the display and then altering the form of the cursor in order toeffect a control and/or operating command with respect to the element; acontrol command or input is effected by pointing the light cursor at aspot on the display in order to highlight an element on the display andthen altering the form of the cursor in order to effect a control and/oroperating command with respect to the element; the cursor is madevisible by highlighting an element on the display by means of a controlsoftware for the display.